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ABSTRACT 


This thesis presents a computer assisted, comparative analysis of 
empirical, maximum likelihood and exponential procedures for estimating 
reliability. The deviations of the estimators from the true reliability, 
when the underlying failure rate is monotone, are compared using the 
Kolmogorov-Smirnov family of statistics. The behavior of the distribu- 
tions of these deviations, for various sample sizes and failure rates is 
examined. Finally, the considerations of when to use which estimator 


are discussed. 
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I, INIRODUCTION 


Assumptions concerning the form of the underlying failure dis- 
tribution are made in most analyses of reliability problems. When 
these assumptions are in error the conclusions reached may be gross- 
ly in error. 

Some alternatives to making assumptions about the form of the 
underlying failure distribution are maximum likelihood estimation and- 
empirical estimation. 

This work presents a comparative analysis of empirical, maximum 
likelihood, and the often used exponential procedures when the under- 
lying failure distribution has a monotone non-decreasing failure rate. 
The maximum likelihood estimator is formed under the assumption that 
the underlying failure rate ae monotone non-decreasing. The hypothesis 
that a sample of failure times comes from a distribution having a mono- 
tone failure rate can be tested using the method contained in Ref. l. 

The relative quality of the three estimators was examined using 
the Kolmogorov-Smirnov family of statistics. 

The following notation and definitions are used in the succeeding 
sections. Let F be a right continuous distribution such that F(0°)=0. 


if has a density £ then r(t)= f(t) is the failure rate, where 


R(t) 
R(t)=1 - F(t) is the reliability or the survival probability. Thus 
t 
R(t) = exp -f ahaa] . A distribution has an increasing failure rate 
~—@ 





(IFR) if r(t) is monotone non-decreasing in t, and has a decreasing 


failure rate (DFR) if r(t) is monotone non-increasing in t. 





II. THE ESTIMATORS 


A. THE EMPIRICAL ESTIMATOR 
The empirical estimator of reliability, Remp(t), based on a 


sample of n ordered observations (t,<t <a -St,) from a failure dis- 


Z 


trébution” Fis; 


Remp (t) =" 1.0 t<t, 
Remp(t) = (n-i)/n ts t <tiy 
Remp(t) = 0 t zt. 


B. THE EXPONENTIAL ESTIMATOR 

The exponential estimator of reliability based on the above sample 
is Rexp(t) = exp [-Kt | , t>0, where 1/k is the maximum likelihood 
estimate of the mean of the exponential distribution and is equal to the 


sample mean. 


C. THE MAXIMUM LIKELIHOOD ESTIMATOR 

The derivation of the maximum likelihood estimator is dependent 
upon an assumption that the underlying distribution has IFR or DFR. 
The IFR case is outlined here. Both the IFR and DFR cases are pre- 
sented in detail in Ref. 1. 

Let t,s tis Apes st, be a sample of n ordered observations from 
F, IFR. Using the fact that 





t 


R(t) = exp -f rle)ee| ance (t)a= a the log likelihood may be 
- 00 


expressed as 


n n n 
Ee 
L = log(n!) + S log f(t,) = YS logr(t,) >) | r (z)dz+log(n!). 
i=1 i=] i=-1 ° 


The maximization of L, subject to r(t) monotone non-decreasing, 


yields as an estimator for r(t) 


-1 
i. ee v= | oma - aoe Seis a (n-v+1)(t, - ta1)| 
i=1,2,...,n-1 and r(t,) =CO. For the remaining values of t, T(t) is 0 
for 0<t< ty ‘tor t >t and constant (right continuous) between 
observations. The corresponding estimator, R(t), is obtained from 


n t 
R(t) = exp -f Medez| . Reference 1 shows that T(t) is a consistent 
al © @) 


estimator of r(t). 





III. PROCEDURES 


To evaluate the estimators of R(t), computer simulation (Fortran IV, 
IBM-360, W. R. Church Computer Center, Naval Postgraduate School) 
was used to generate samples of failure times, compute the estimators 
and..the..statistics used in their evaluation. 

Two parent distributions were used, the Weibull and the Erlang. 
The Weibull distribution has reliability function R(t) = @ bt® with shape 
Parameter a>O and scale parameter b>0O. The failure rate of the 
Weibull distribution is r(t) = af When a=1 the distribution is the 
exponential. For values of az>l_ the Weibull distribution has IFR. 

The Erlang distribution was used to investigate whether the results 
were dependent upon the parent distribution used. The Erlang distribu- 

cele) peal aoe 
tion has reliability function R(t)= - (= near 


and a is aninteger. The failure rate of the Erlang distribution is 


dx where a>0O, b>0O 


ta-1 pa _7 Pt 
me) = i Sa . When a=1 the distribution is the exponential. 
When a21_ the Erlang distribution has IFR and the failure rate is 
bounded above by b. 

Consider a sample of n independent, identically distributed 
realizations of Weibull failure times. The true reliability of a system 
with this underlying distribution is R(t) =e7bt? . Let Remp(t) be the 


empirical estimator of R(t) as computed from this sample and let 


Then Demp is a one-sample, two-sided, 





Demp=sup | Remp(t) - R(t) 
5 | 








Kolmogorov-Smirnov statistic. Computing Demp for each of one 
hundred independent samples and ordering the results yields an empiri- 
cal distribution of the statistic Demp. 

Repeating this procedure for each estimator gives empirical distri- 
Rexp(t) - R(t) 


butions for the statistics Dexp= sup and 








Dmle=sup /|Rmle(t) - R(t) Let Fdemp(x) be the empirical distri- 


t 
bution function of the statistic Demp. Similarly, let Fdexp(x) and 








Fdmle(x) be the empirical distribution functions of Dexp and Dmle 
respectively. 
For a given set of Weibull distribution parameters and a given 


sample size, define; 


De = sup | Faemp (x) - Fdmle(x) | 
D = sup | Famle (x) - Fdemp(x) | 
Dy = sup | Famle(«x) ~ Fdexp(x) | 
DF = sup | Paexp(x) - Fdmle(x) | 
D; = sup | Faemp(x) - Fdexp(x) | 


D3 = sup | Faexp (x) - Fdemp(x) | 
x 


These statistics are two-sample, one-sided, Kolmogorov-Smirnov 
statistics and can be used to test the following types of hypotheses; 
Ho: Fdemp(x) = Fdexp(x) against the alternative H1: Fdemp(x) > Fdexp(x). 

To define the critical region for this test, note that 

+ + + 
P [D3 =) (0) | =1-&,where D (&) is the 1- % percentile of the 


two-sample, one-sided, Kolmogorov-Smirnov statistic distribution. 





Reference 2 gives an approximation to the limiting distribution of Das 


2 
n7+n Es 
lim P D<x elie = ]-X= l-e ox where n. and n. are 
n j njn, il 2 


Dee 


n.~->0oo 


Z 
the sample sizes of the two distributions tested. 

For a test of the hypothesis Ho: Fdemp(x) = Fdexp(x) against the 
alternative H1: Fdemp(x) 4 Fdexp(x), the two-sample, two-sided 
Kolmogorov-Smirnov test may be used. The statistic is D,=max D3. D3 | : 


Reference 3 defines the limiting distribution of D and Ref. 2 gives the 


following approximation 


2 
+ = 
lim BaD <x ee =]-~c=1-2e — 
nj>0 nyjyng 


Wis 


for values of x sufficiently large. 
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IV. RESULTS 


The failure rates examined vary from r(t) = 1.0, the case when the 
underlying distribution is exponential, to r(t)=2. 139t? . In each case 
the mean time to failure is equal to one. 

The important difference in these failure rates is their relative 
rate of change ie t. Throughout the remainder of this discussion 
the term high failure rate refers to a failure rate with a relatively high 
rate of change with t. Thus coete is a higher failure rate than 
r(t)=t. 

Table I contains the statistics D, and Dh , as defined in 
Section III. The critical values of these statistics, at a level of 
significance of .01, are D’ ,, (.01) = De, (-ON=0.215. At a level 
of significance of .05 D’_.,(.05)=D__,,(.05) = 0.173. The statistic 
D can be determined from the table by using the fact that D= max [p* >| , 
The critical values of D at levels of significance of .01 and .05 are 
Dt OU = 0.230 and D yt (05) = 0-192. 

At a level of significance of .01 the hypothesis Ho: Fdmle(x) = 
Fdemp(x) is rejected in favor of Hl: Fdemp(x) # Fdmle(x) when 
D> 0.230. Ho is rejected in favor of H1: Fdemp(x) > Fdmle(x) when 
Dy > 0.215. 

Reversing the sense of the inequality in Hl and replacing iy by 
Dy the test becomes, ee Ho in favor of Hl: Fdmle(x)> Fdemp(x) when 


D\> OF215. 


ll 





As all samples of Dexp, Demp and Dmle are of the same size 
and theKolmogorov-Smirnov statistics are independent of the distribu- 
+ -_ 
tions being tested, these critical values of D and D can be used 


for all cases presented here. 


TABLE I 


Observed values of D} = sup [Pacmptx) -Famle(s) and 
x 


Dj= sup [ramtets) - Faemp(x) for sample size n and failure rate r(t) 
X 





In all cases presented in Table I the value of Dy is zero and the 


value of Di is significantly large which implies that the empirical 
estimator deviates less from the true reliability than does the maxi- 
mum likelihood estimator. 

As the sample size increases, it is expected that the deviations 
of both estimators will decrease. The data in Table I indicate that 


the deviations of Remp(t) are decreasing faster than the deviations of 


Rmle(t). 
12 





Before examining some of the sample points of the distributions 
Fdemp(x) and Fdmle(x), it should be noted that the sample ipa 
percentile of Fdmle, denoted by Dmle(100), is the maximum devia- 
tion of Dmle for all samples of fixed size n with the same under- 
lying failure rate. Similarly, Dmle(1) is the minimum deviation and 
Dmle(100) -Dmle(1) is the sample range of Dmle. 

As Demp is a one-sample, two-sided Kolmogorov-Smirnov sta- 
tistic, it is independent of the distributions tested and will vary 
only with sample size. This is not true of Dexp and Dmle. 

When the failure rate is 2. 1397 and the sample size increases 
from 10 to 30, Demp(100) decreases from .4120 to .2690, a reduction 
of about 35% in maximum deviation. Over the same range of sample 
sizes Dmle(100) decreases from .5426 to .4470, a reduction of about 
15% in the maximum deviation. For the same changes in sample size, 
Dmle(1) varies from .1816 to .1810 while Demp(1) varies from .1210 
to .0671. This implies that the entire range of Demp moves toward the 
origin as the sample size increases, while only the right hand coal 
of Dmle moves with changes in sample size. When the failure rate is 
constant the variation in Dmle(100) is from .6956 to .6165 while the 
variation in Dmle(1) is from .1974 to .1899. 

The data in Table II indicate that the deviations of the expon- 
ential estimator are stochastically smaller than the deviations of the 


maximum likelihood estimator for all failure rates tested. 


13 





TABLE II 


Observed values of D, = sup [Famle (x) - Fdexp(x) | and 
x 


D. = sup [Faexp(x) - Famle (x) for sample size n and failure rate r(t) 
x 





For a given failure rate, the magnitude of D, increases with 
sample size and for a fixed sample size it decreases as the failure 
rate Mera eee 

The behavior of the distributions of Dexp and Dmle are best 


illustrated graphically. Figures 1 through 4 depict a smoothed form 


of the empirical distributions as sample size and failure rate vary. 
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Figs i 


Note: 


Fig. 2. 


Note: 


Fdexp Fdmle 


ae 24 <6 HB, 


Fdexp and Fdmle for n = 30 and r(t) = 1.0 


Dexpil) = 0.0001 Dexp (100) = 0.1651 
pmle (1) = 0, 1899 Dmle (100) = 0.6165 
Fdexp Fdmle 

ae 4 26 D 
Fdexp and Fdmle for n = 30 and r(t) = 2. 139¢7 
Dexp (1) = 0.2610 exp (100) = 0.3565 
Dmle) = 0.1914 Dmle (100) = 0.4473 
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F Fdexp Fdmle 


0 eZ 4 .6 D 
Fig. 3. Fdexp and Fdmie for n= 10 and r(t) = 2. 139t? 
Note: Dexp (1)=0.2399 Dexp (100)=0.4291 


Dmle (1)=0.1816 Dmle (100) =0.5462 


1.0 
Fdexp 


Fdmle 


0 we 4 6 D 


Fig. 4. Fdexp and Fdmle for n= 10 and r(t) = 1.0 
Note: Dexp it) — 07,0010 Dexp (100)=0.3571 


Dmle (1)=0.1979 Dmle (100 )= 0.6456 





As would be expected, the exponential estimator performs best 
when the failure rate is low and the sample size is large. As the 
failure rate increases Fdexp moves to the right and the range of Dexp 
decreases. 

The range of Dmle decreases as sample size increases or as 
failure rate increases. The first percentile of the distribution is rela- 
tively insensitive to change in either sample size or failure rate. 

For all cases, Ho: Fdmle(x) = Fdexp(x) can be rejected in favor 
of Hl: Fdexp(x) > Fdmle(x) at level .01. 

The data in Table ITI indicate that the exponential estimator 
deviates less from the true reliability than does the empirical esti- 
mator when the failure rate is constant or nearly constant. When the 
failure rate is constant, D, = 0 and D, increases with the sample 
size, n. This is consistent with the theory as the maximum likeli- 


hood estimate of the parameter of the exponential distribution is the 


reciprocal of the sample mean and lim x= MM, 
>@ 
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TABLE III 


+ 
Observed values of D, — abhe) | Fdemp(x) - Fdexp(x)| and 
x 


D.=sup [Fdexp(x) - Fdemp(x)| for sample size n and failure rate r(t) 
x 





As the failure rate De OSI larger, D, decreases to zero. D 
increases with both sample size and failure rate. When the failure 
rate is not constant, D, decreases with sample size. 

Figures 5 through 8 show the behavior of Fdemp and Fdexp with 


changes in failure rate and sample size. 
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Ihe 
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0 


0 


Fdexp 


Fig. 9. 


Note: 


rig. (6.. 


Note: 


Fdemp 
2 4 =e D 
Fdexp and Fdemp forn= 1.0 and r(t)=1.0 
Demo) — 0.000 Dexp (100) =0.357] 
Demp (1)= 0.1207 Demo (100)= 074 122 
Fdemp Fdexp 
eZ, 4 AS D 
Fdexp and Fdemp for n = 10 and r(t) = 2.139 tz 
Dexp (1) = 0.2399 Dexp (100) = 0.4291 


Demp (1)= 0.1207 Demp (100) = 0.4122 





Fdexp 


0 
F Fdemp 
0 ae 24 a0 D 
Fig. 7. Fdexp and Fdemp for n= 30 and r(t) = 1.0 
Note: Dexp (1) = 0.0001 Dexp (100) = 0.1651 
Demprtl)=— 0.0670 Demp (100)= 0.2690 
10 
Fdemp Fdexp 
F 
0 2 24 .6 D 


Fig. 8.  Fdexp and Fdemp for n= 30 and r(t)=2.139 ie 


Note: expel = 0026 10 Dexp (100) = 0.3565 


Demp (1) = 0.0670 Demp (100) = 0.2690 


Note that the distribution Fdemp is invariant with changes in 


failure rate and varies only with the sample size. The changes in 


of ~ 
D, and D, for a given sample size occur because the distribution 


Fdexp varies with r(t). As r(t) increases Fdexp moves to the right 


20 





for all sample sizes examined. For high failure rates the left end. 
point of Fdexp moves away from the origin as sample size increases 
while the right end point moves toward the origin. 

The stochastic ordering of Fdexp and Fdemp varies with the 


underlying failure rate with Fdemp >Fdexp for higher failure rates. 


ea 


Zi 
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VV. CONCLUSIONS 


The statistics in Tables I and II support the hypothesis that the 
empirical and the exponential estimators give better estimates of 
reliability than does the maximum likelihood estimator for the cases 
considered. 

Comparing the exponential and the empirical estimators, it can 
be seen that while both sample size and failure rate affect the mag- 
nitudes of D* and D , the failure rate of the underlying distribution 
is the factor that determines the stochastic ordering of the distributions 
Fdemp and Fdexp. 

The exponential estimator would be the logical choice when the 
underlying failure rate is constant, or nearly constant, while the 
empirical estimator performs better when the failure rate is higher. 

In considering the question of when to use which estimator it 
is necessary to investigate the nature of the underlying failure rate. 

Reference 4 contains several tests for the validity of the assump- 
tion that the underlying life distribution is exponential. 

If physical considerations or tests, such as those presented in 
Ref. 4 substantiate the assumptions that the underlying distribution 
is exponential, then the exponential estimator of reliability should be 
used. If the exponential assumption cannot be substantiated, the 
empirical estimator should give better estimates of the system reli- 


ability than the exponential estimator. 


Ze 





Physical considerations of some systems indicate that the failure 
rate is not monotone but conforms to the bathtub model where the 
failure rate is initially high but decreasing, relatively constant for 
the middle age life periods, and increasing with old age. Some 
electronic components are believed to behave in this manner, experi- 
encing a "burn-in" period at the onset of their working life cycle. 

The empirical estimator should be particularly useful in cases 
where the failure rate is not monotone as no assumptions about the 


form of the underlying distribution, or its failure rate are required. 
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